Use endpoint IDs for endpoint predict requests#78
Use endpoint IDs for endpoint predict requests#78phil-scale wants to merge 1 commit intolaunch-v1from
Conversation
| `abcabcab-cabc-abca-0123456789ab` | ||
| """ | ||
| validate_task_request(url=url, args=args) | ||
| endpoint = self.get_model_endpoint(endpoint_name) |
There was a problem hiding this comment.
is this just a postgres call?
There was a problem hiding this comment.
Yes, but also Redis to get some cached data about the infra state (and k8s api server if Redis cache doesn't contain the info)
yixu34
left a comment
yixu34
left a comment
There was a problem hiding this comment.
Wait is this a breaking change?
Sort of? |
True, though I think enough people might have latched onto Maybe we can do a broader change as part of #73? |
|
Ok closing this in favor of modifying #73 instead |
3 participants
Extra latency right now for sync/async requests incurred from querying for endpoint by name